This application is related to commonly owned U.S. patent application Ser. Nos. 09/738,941 and 09/738,564, both filed on Dec. 15, 2000, and respectively entitled SYSTEM AND METHOD FOR SERVING OF ROBOTS USING FIDUCIAL MARKS AND MACHINE VISION AND SYSTEM AND METHOD FOR LOCATING AN OBJECT IN THREE DIMENSIONS BASED UPON MULTIPLE FIDUCIALS USING MACHINE VISION both by Aaron S. Wallack, and expressly incorporated herein by reference.
1. Field of the Invention
This invention relates to machine vision systems and more particularly to uses for advanced machine vision search tools that identify and register patterns transformed by at least two translational and at least one non-translational degree of freedom.
2. Background Information
In the past, there have been several tools such as three dimensional cameras, which have been able to acquire range images, a digital image where the values of the pixels represent the distance from a point or plane fixed with respect to the camera tool. This range image data, however, has not been employed with machine vision systems utilizing rotation/scale-invariant (RSIS) search tools. Current search tools are used in conjunction with cameras that acquire images formed of grey level data. This grey level data, however, does not always accurately or precisely identify movement of an object towards or away from the camera or the distance of the object from the camera.
The use of advanced machine vision systems and their underlying software is increasingly employed in a variety of manufacturing and quality control processes. Machine vision enables quicker, more accurate and repeatable results to be obtained in the production of both mass-produced and custom products. Basic machine vision systems include one or more cameras (typically having solid-state charge couple device (CCD) imaging elements) directed at an area of interest, frame grabber/image processing elements that capture and transmit CCD images, a computer and display for running the machine vision software application and manipulating the captured images, and appropriate illumination on the area of interest.
Many applications of machine vision involve the inspection of components and surfaces for defects that affect quality. Where sufficiently serious defects are noted, a part of a surface is marked as unacceptable/defective. Machine vision has also been employed in varying degrees to assist in manipulating manufacturing engines in the performance of specific tasks. One task using machine vision is visual serving of robots in is which a robot end effector is guided to a target using a machine vision feedback. Other applications also employ machine vision to locate a stationary and/or moving pattern.
The advent of increasingly faster and higher-performance computers, has enabled the development of machine vision systems that employ powerful search tools. Such search tools enable a previously trained/stored image pattern to be acquired and registered/identified regardless of its viewed position. In particular, existing commercially available search tools can register such patterns transformed by at least three degrees of freedom, including two translational degrees (x and y-axis image plane) and at least one non-translational degree (rotation and/or scale, for example). One particular implementation of an advanced search tool is the rotation/scale-invariant search (RSIS) tool. This tool registers an image transformed by at least four degrees of freedom including the two translational degrees (x and y-axis image plane) and at least two non-translational degrees (scale) and rotation within the x-y plane about an axis perpendicular to the plane. Some tools also register more complex transformations such as aspect ratio (rotation out of the plane whereby size on one axis decreases while size in the transverse axis thereto remains the same). These search tools, therefore, enable a specific pattern within the field of view to be located within a camera field of view to be positively identified and located accurately within the vision system""s internal reference system (an x, y, scale, rotation coordinate system, for example). The RSIS and other advanced search tools particularly allow for the identification and acquisition of patterns having somewhat arbitrary rotation, scaling and translation with respect to the reference system. In other words, the tool is sufficiently robust to recognize a desired pattern even if it is rotated and larger/smaller/skewed relative to a xe2x80x9cmodelxe2x80x9d or trained pattern within the vision system.
In general, advanced machine vision tools acquire an image of a pattern via a camera and analyze the outline or a particular part of the pattern, such as a predetermined fiducial mark. The processing speed of the underlying computer in which the tool resides is sufficient to enable a very large number of real time calculations to be completed in a short time frame. This particularly enables the search tool to determine the coordinates within an image reference system for each analyzed point in the viewed area, and correlate these through repetition with a desired pattern. The search tool may map the locations of various points in the captured image stored points in the model image, and determine whether the captured image points fall within an acceptable range of values relative to the model image points. Using various decision algorithms, the tool decides whether the viewed pattern, in a particular rotation and scale corresponds to the desired search pattern. If so, the tool confirms that the viewed pattern is, in fact, the pattern for which the tool is searching and fixes its position and orientation.
Machine vision systems having a three-degree-of-freedom, or greater, capability (such as RSIS) are available from a number of commercial vendors including Hexavision(copyright) from Adept Technology, Inc. of San Jose, Calif., and the popular Patmax(copyright) system from Cognex Corporation of Natick, Mass. Advanced machine vision search tools such as Patmax(copyright) also have the ability to take advantage of the previous known position of a search subject or target. This narrows the search area to positions relatively near the last known location. Therefore, searching is relatively faster on the next cycle since a smaller area is searched. In addition, these search tools can tolerate partial occlusion of a pattern and changes in its illumination, adding further to their robustness with respect to less advanced machine vision approaches.
One goal of machine vision is to provide a more accurate technique for identifying and locating objects in three dimensions. In the past, as was described above, the search tools have used an image acquired by cameras which generally consist of a plurality of pixels. Each pixel has a value corresponding to light intensity. Though in most cases, the pixels allow the machine vision search tool to identify and register patterns, the tools may, in certain situations, not be as accurate, as possible, in registering the pattern in terms of scale. In particular, systems that measure the relative scale of the pattern which is often accurate only to 0.1% of measured scale, while transformation within the image plane (x,y) is often accurately measured to within {fraction (1/40)} Pel.
Accordingly, it is an object of this invention to provide a system and method to more accurately identify and register an object using a machine vision system utilizing range data images.
This invention overcomes the disadvantages of the prior art by providing a system and method for locating and registering a pattern in a range image of an object. The invention uses an advanced machine vision search tool capable of registering an acquired pattern in a range image transformed by at least three degrees of freedom including, generally, at least two translational degrees of freedom and at least one non-translational degree of freedom with respect to an image plane (e.g. x and y-axis plane) as viewed by the machine vision system. The machine vision search tool can be a rotation/scale-invariant search (RSIS) tool, or another equivalent implementation adapted to recognize and locate one of more previously trained patterns regardless of translation and rotation of the pattern within the specified degrees of freedom (four or more in this example).
The range image is preferably acquired by a three-dimensional camera which the value of such pixel represents an accurate distance from the camera to the corresponding point on the object""s surface or distance from a reference plane or point fixed with the respect to the camera to an objects surface.
In one embodiment, the system is used to determine if a run time object has moved relative to a training object with respect to the reference plane or point discussed above. The camera preferably also obtains a confidence value for each range pixel. In a preferred embodiment, the values of the pixels are adjusted or smoothed to compensate for low confidence values or depth discontinuity boundaries to produce a synthetic range image. This adjustment uses the confidence values and range and confidence values of the neighboring range pixel to replace inaccurate range pixel values.
In one embodiment, inaccurate range pixel range image measurements are determined and masked from being used in constructing the synthetic range image.